The present invention relates to a diagnostic method for a vision sensor of a vehicle and to a vehicle having a vision sensor.
Modern vehicles, in particular motor vehicles, have a multiplicity of driver assistance systems which can assist the driver or relieve the pressure on the driver during his driving activity. Vehicles which can carry out some driving tasks entirely or partially without the intervention of the vehicle driver are known. The operation of such vehicles has become known under the term partially or fully automatic or partially or fully autonomous driving. These driver assistance systems require the most comprehensive possible image of the traffic situation surrounding the vehicle. For this purpose, modern vehicles have a multiplicity of vision sensors which can detect the area surrounding the vehicle. This makes it possible to sense and detect objects in the area surrounding the vehicle. Such objects may be other vehicles, pedestrians, cyclists, traffic signs, structures and the like. Movable and immovable objects may be detected.
In order to guarantee reliable operation of the driver assistance systems, the functionality of the vision sensors must be ensured. However, this may be impaired. On the one hand, the vision sensors as such may be defective or may have restricted functionality. On the other hand, the functionality of a vision sensor may be restricted by external influences. Examples of such external influences are, for example, fog, rain and a soiled sensor outer surface. So that a driver assistance system which uses data provided by a vision sensor in turn functions properly, it should be ensured that the data provided by the vision sensor are not corrupted by a functional incapacity of the type mentioned above. It has therefore been proposed to carry out methods for identifying a restricted functionality of vision sensors. If it is identified during such a method that the functionality of the vision sensor is restricted, the driver assistance system can reject the data provided by this vision sensor or can attribute a reduced importance to them during data processing or can take further suitable measures.
A method for operating a sensor of a vehicle is known from DE 10 2010 049 091 A1. In this case, position data relating to an object are first of all received and a position of the object is determined. If this position is in a predetermined detection range of a sensor of the vehicle, it is determined whether the object is detected by the sensor. If this is not the case, a warning message can be output and/or a driver assistance system can be adjusted.
The disadvantage of the method known from the prior art is that it can only be carried out with objects which are set up to transmit their position data. The object must therefore be set up for data transmission, which does not apply to a multiplicity of objects. Furthermore, the method may be inaccurate when the object is a movable object on account of the movement of the object. A further disadvantage is that the vehicle requires a receiving device for receiving the position data transmitted by the object in order to carry out the method.
On the basis of the prior art, the object of the invention is to provide a diagnostic method for a vision sensor of a vehicle, which diagnostic method can be used in as versatile a manner as possible, can be easily carried out and is reliable. The object is also to provide a vehicle having a vision sensor which is suitable for carrying out such a diagnostic method.
These and other objects are achieved in the case of a diagnostic method and a vehicle in accordance with embodiments of the invention.
In a first step of the method according to the invention, a first position and a direction of movement of the vehicle are determined. The first position is that geographical position at which the vehicle is situated at the time of carrying out the method step. The process of determining the first position preferably comprises determining the geographical coordinates (geographical latitude and longitude) of the first position. Apparatuses of the vehicle which are known per se can be used to determine the first position and/or the direction of movement of the vehicle, in particular a satellite navigation system of the vehicle. The direction of movement of the vehicle can be determined as a direction vector. This direction vector can be determined with reference to a reference coordinate system (for example world coordinate system). The direction of movement of the vehicle can also be determined with reference to an object of the traffic infrastructure. For example, if the vehicle is traveling on a road, the direction of travel with reference to the road being traveled on can also be stated as the direction of movement. For example, such a statement could be “Route 1 in a northerly direction” or “Freeway A9 in the direction of Munich”.
In a subsequent method step, a stationary landmark suitable for detection by the vision sensor is selected from a landmark database. A stationary landmark may also be referred to as an immovable landmark. The stationary landmark is therefore an object which is always at the same location. The landmark database is a database which stores information relating to landmarks. The contents stored in the landmark database may have been acquired and provided by a provider, for example. The landmark database may contain a multiplicity of items of information for a particular landmark. At least the type of landmark and its geographical position must be stored in the landmark database for each landmark which is suitable for carrying out the method according to the invention. On the basis of the information relating to the type of landmark, it is possible to determine whether the landmark is suitable for detection by the vision sensor. The geographical position of the landmark must be known so that the landmark can be selected, as described above, in such a manner that it is in front of the vehicle in the direction of movement. Moreover, the information relating to the geographical position is required in the further course of the method.
Three conditions are taken into account when selecting the landmark. Firstly, the landmark must be suitable for detection by the vision sensor. If the vision sensor is a radar sensor, for example, the landmark must be a landmark which can be detected by a radar sensor. In contrast, a landmark which cannot be detected by a radar sensor on account of its nature is not suitable in this example. Secondly, the landmark must be in front of the vehicle in the direction of movement of the vehicle. In other words, a landmark toward which the vehicle is moving is selected. It is not necessary for the vehicle to move toward the landmark head-on. The landmark therefore need not be directly in front of the vehicle in the direction of movement. It is only necessary for the distance between the vehicle and the landmark to be reduced during movement of the vehicle in the direction of movement. The landmark may therefore be a landmark at the side of the road which will be passed by the vehicle during its journey (that is to say its movement in the direction of movement). Thirdly, a distance between the first position of the vehicle and the landmark must be greater than the desired range of the vision sensor. In other words, the landmark is selected in such a manner that it cannot yet be detected by the vision sensor at the time at which the vehicle is at the first position because it is outside the detection range of the sensor. However, the landmark is selected in such a manner that it will enter the detection range of the vision sensor at a later time at which the vehicle has already moved further in its direction of movement. A course of the road ahead (for example a road course) can preferably be taken into account when selecting the landmark. This makes it possible to ensure that the selected landmark can actually be detected by the vision sensor over the further driving course.
In a subsequent method step, the landmark is detected by the vision sensor. This step is carried out continuously or repeatedly until the vision sensor has successfully detected the landmark. As explained above, the landmark is initially still outside the detection range of the vision sensor. Since the step of detecting the landmark by means of the vision sensor is carried out continuously or repeatedly while the vehicle is moving in the direction of the landmark, the vision sensor will successfully detect the landmark as soon as the landmark enters the actual detection range of the vision sensor (determined by the actual range of the vision sensor).
Provision may preferably be made for the method to be aborted and started from the beginning if the vehicle has passed the landmark (that is to say the distance between the vehicle and the landmark increases again) but the landmark could not be successfully detected.
In response to detection of the landmark by the vision sensor (that is to say as soon as the landmark has been successfully detected by the vision sensor), a step of determining a second position of the vehicle is carried out according to the invention. In other words, the position of the vehicle at the time at which the landmark was successfully detected by the vision sensor is determined. In a final method step, the actual range of the vision sensor is determined as the distance between the second position of the vehicle and the landmark.
The method according to the invention therefore makes it possible to determine the actual range of the vision sensor in a simple manner by using the fact that the exact position of the landmark to be detected and detected by the vision sensor is known a priori. As soon as the vision sensor successfully (that is to say actually) detects the landmark, the distance between the vision sensor and the landmark corresponds to the actual range of the vision sensor.
Embodiments of the invention provide for the vision sensor to be a camera, a laser sensor, a radar sensor, an ultrasonic sensor or a bolometer. The term laser sensor should be broadly understood and also comprises, in particular, a lidar sensor. Vision sensors of the type mentioned are used, in particular, in motor vehicles in order to detect objects in the area surrounding the motor vehicle. Modern motor vehicles having a partially automatic or fully automatic driving mode may have numerous vision sensors of different types which continuously detect the area surrounding the motor vehicle. The same landmark can be used, in a particularly advantageous manner, to determine the respective actual ranges of a plurality of vision sensors. For this purpose, the method is carried out for each of the plurality of vision sensors, the same landmark being selected in each of the methods. For this purpose, the landmark must be suitable for detection by each of the vision sensors. If, for example, a lidar sensor and a radar sensor of a vehicle are intended to be tested using the same landmark, the landmark must be suitable both for detection by a lidar sensor and for detection by a radar sensor.
Provision is made, in a particularly advantageous manner, for the landmark to be a coded landmark. A coded landmark is a landmark which cannot only be detected by a vision sensor but has at least one coding feature which is used primarily or solely for the better detectability of the landmark by a vision sensor. For example, numerous objects of the traffic infrastructure (road signs, traffic signs, road markings, structures, for example bridges) are suitable for detection by vision sensors, in particular cameras and lidar sensors. However, these are generally not coded landmarks. In contrast, a coded landmark has a coding feature. A coded landmark may be an object of the traffic infrastructure (for example road signs, traffic signs, road markings, structures, for example bridges) which, in addition to its actual purpose, has been provided with a coding feature. Alternatively, a coded landmark may also be an object which is used solely as a coded landmark. A coding feature of a coded landmark may be such that it can be detected particularly well by a vision sensor of a particular type (for example camera, radar sensor, lidar sensor, ultrasonic sensor). A coding feature may be such that it can be detected particularly well by vision sensors of different types. In other words, a coding feature may be selected in such a manner that the coded landmark comprising the coding feature is suitable for detection by vision sensors of different types, for example both by a camera and by a lidar sensor. The configuration of a coding feature, in particular with regard to its shape, its color, its acoustic properties and/or its electromagnetic properties (including its optical properties), may be such that it can be detected particularly well by the vision sensor, on the one hand, and makes it possible to distinguish the coded landmark from other objects, on the other hand. In other words, the coding feature may be configured in such a manner that the coded landmark differs to the greatest possible extent from other objects, with the result that random confusion of the coded landmark with another object is as unlikely as possible. A coding feature may comprise an item of information which can be concomitantly acquired by the vision sensor when detecting the coded landmark (and therefore when detecting the coding feature). Such an item of information may be a unique identifier of the coded landmark, for example. Such an item of information may be, for example, a statement of the geographical position of the coded landmark.
For detection by active vision sensors, retroreflective coding features are advantageously suitable, which coding features have, in a particularly advantageous manner, a depth structure in order to enable detection from different directions. Active vision sensors are those vision sensors which emit a signal. The active vision sensors include, for example, radar sensors, lidar sensors and ultrasonic sensors. Passive vision sensors can be distinguished from active vision sensors. Passive vision sensors are set up to receive a signal without previously having emitted their own signal. The passive vision sensors include cameras, for example. Coding features which, on account of their configuration, cannot be confused with objects randomly occurring in the environment and, in particular, comprise an item of information, for example a unique identifier, are advantageously suitable for detection by passive vision sensors.
The size of the coding feature may be advantageously selected on the basis of the desired range and/or a resolution of a vision sensor suitable for detecting the coded landmark.
Provision may be made for coded landmarks to preferably be selected when selecting the landmark. If, during the step of selecting the stationary landmark suitable for detection by the vision sensor from the landmark database, it is therefore possible to select from a plurality of landmarks, the step may comprise preferably selecting a coded landmark.
With further advantage, the landmark is a passive landmark. In contrast to an active landmark, a passive landmark is a landmark which is not set up to actively emit a signal. The method according to the invention is not dependent on the landmark emitting a signal which can be used to identify it as a landmark and which possibly contains the position of the landmark. The method can be used in a versatile manner by using passive landmarks for the method. This is because, whereas the installation and operation of active landmarks are relatively complicated, passive landmarks can be used in large numbers. In particular, all suitable objects of the traffic infrastructure can be used for the method according to the invention without further measures if they are recorded in the landmark database, as described above.
In a further configuration, an uncertainty range of the first position of the vehicle and/or an uncertainty range of a position of the landmark is/are additionally determined. The uncertainty range of the first position of the vehicle may be determined, for example, by that apparatus of the vehicle which determines the first position of the vehicle, that is to say by a satellite navigation system of the vehicle, for example. Said satellite navigation system can determine, for example in a manner known per se, an uncertainty range on the basis of the number of satellites which could be used during position determination and on the basis of the signal quality of the signals received from the satellites. The uncertainty range may include, for example, a statement of a radius around the first position within which the actual position of the vehicle is located with a predefined probability, for example 90% or 95%. The uncertainty range of the position of the landmark may be recorded in the landmark database, for example. Provision is also made for the step of selecting the landmark to be carried out in such a manner that a distance between the first position of the vehicle and the landmark is greater than the sum of the desired range of the vision sensor and the uncertainty range of the first position of the vehicle and/or the uncertainty range of the position of the landmark. In other words, the landmark is selected in such a manner that it is outside the desired range of the vision sensor even taking into account the uncertainty range(s). This ensures that the landmark only enters the detection range of the vision sensor during the further course of the vehicle movement (and therefore in the further course of the method).
A particularly advantageous development of the invention provides for the diagnostic method according to the invention to be performed repeatedly, that is to say a first time and at least one further time. In this case, a landmark which differs from the landmark selected and detected during the previous performance of the method is selected and detected during each further performance of the method. A mean value of the actual range is finally determined from the at least two determined actual ranges. In other words, provision is therefore made for the method to be carried out repeatedly and for a mean value to be formed from the respective results. This makes it possible to increase the reliability of the determined actual range by reducing the influence of possible error sources by forming the mean value. Possible error sources may include errors and/or inaccuracies in the landmark database and/or when determining the first and/or second position of the vehicle.
A vehicle according to the invention has a vision sensor with a desired range, a navigation apparatus for determining a position and a direction of movement of the vehicle, a data apparatus for accessing a landmark database for storing at least one position of at least one stationary landmark, and a diagnostic apparatus for carrying out the method according to the invention.
One embodiment provides for the landmark database to be part of the vehicle. For example, the landmark database may be stored in a data memory of the navigation apparatus or of the diagnostic apparatus.
An alternative embodiment provides for the landmark database to be arranged outside the vehicle, for example on an Internet server. In this embodiment, the vehicle has a communication module for data communication with the landmark database. Such data communication can be carried out using mobile radio (LTE, UMTS and the like) and/or WLAN, for example.
The invention is suitable for all types of vehicles, in particular for motor vehicles, in particular automobiles and trucks, and for motorcycles and for watercraft. In particular, the invention is suitable for vehicles which have a partially automatic or fully automatic driving mode.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings.